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ABSTRACT 

Apache Junction Unified School District, Arizona, has 
embarked on an 5-year program of instructional improvement using technology. 
PLATO Elementary reading and mathematics products were installed in the 
district's four elementary and two middle schools at the beginning of the 
1999-2000 school year. This evaluation studied the use and preliminary 
student outcomes for these PLATO programs: (1) Beginning Reading for the Real 

World; (2) Projects for the Real World; and (3) Math Expeditions. The study 
focused on the use of the PLATO products as' part of a 4-week summer program 
targeted at those who were below grade level expectations in reading and 
mathematics. In classes for grades K through 6, approximately 100 students 
attended at each elementary school and 75 students attended the middle 
schools. Complete data were available for 15 to 20 students at each grade. 

The typical student used PLATO products about 25 minutes each day. All of the 
third grade students scores as at or above the highest third grade pretest 
scores, and analyses for first, second, and fourth grades showed a generally 
positive correlation between the level of program PLATO use and .posttest 
student achievement scores. Students who used PLATO the most progressed the 
most. Overall, the 3 teachers interviewed for the evaluation and the 74 
teachers who completed survey forms were very pleased with the PLATO 
Elementary products, although their comments do point out some problem areas. 
An appendix contains the teacher survey and responses. (Contains 17 figures 
and 6 tables .*.)"■( SLD) 
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Executive Summary 



Apache Junction Unified School District has embarked on an ambitious five-year 
program of instructional improvement using technology. PLATO Elementary reading and 
math products were installed in the district’s four elementary and two middle schools at 
the beginning of the 1999-2000 school year. 

The district went through a rigorous selection process before purchasing the PLATO 
Elementary products. PLATO training has been provided to teachers in the schools over 
the last two years; the training program is ongoing. As part of the installation, Apache 
Junction has identified technology support personnel both at the district level and within 
each school. Teachers use the program as part of their regular classroom instruction. 

This evaluation describes the use and preliminary student outcomes from PLATO 
programs Beginning Reading for the Real World, Projects for the Real World, and Math 
Expeditions. The study focused on the use of the PLATO products as part of a four-week 
summer program targeted to help who were below grade level expectations in reading 
and math. Classes were offered in grades K to 6; approximately 100 students attended 
each elementary school and 75 students attended the middle schools. The summer 
program involved a total of 64 classroom hours of testing and instruction. This study was 
able to collect research data from four of the summer school classes — a first, second, 
third, and fourth grade class. 

During the summer program the typical student used PLATO Elementary programs for a 
little less than six hours over the course of the entire program; this would be about 
twenty-five minutes per day. Students in fourth grade spent somewhat less time on the 
PLATO Elementary programs than students in first, second, or third grades. The number 
of PLATO Elementary activities mastered ranged from about ten activities in the first 
grade to over 35 activities in the fourth grade, more than tripling from first to fourth 
grade. 

An analysis of pretest-posttest scores in the third grade summer school class showed a 
significant and large change in both reading and mathematics achievement. The third- 
grade data showed that all of the students scored as at or above the highest third-grade 
pretest score. Additional analysis for first grade, second grade, and fourth grade classes 
showed a generally positive correlation between the level of PLATO program use and 
posttest student achievement scores: students who used PLATO the most progressed the 
most. 

Overall the educators responding for the evaluation were very pleased with the PLATO 
Elementary products. Results of the educator interviews and surveys point out that the 
positive aspects of PLATO Elementary for Apache Junction were: screen design, lesson 
content, and teacher training. Results also disclose five problematic issues that are 
indicated elsewhere: Software bugs and problems, students being confused or trapped in 
the program, and teacher frustration, especially in terms of lack of computer access and 
the resulting complexity of running student “groups”. Teachers and district administrators 
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provided details of how the PLATO Elementary program was used. They also identified 
many program strengths as well as specific suggestions for improving its design and 
implementation. 



Apache Junction , AZ 



Page 3 



Table off Contents 



Executive Summary 

Introduction 

Program Description 

The District 

Technology in the District 

Software Selection 

Support of PLATO Elementary 

Teacher Preparation 

Program Design 

Student Placement in PLATO Elementary 

Student Computer Use 

Monitoring Student Achievement 

Learning Materials 

Data Analysis 

PLATO Elementary Use and Test Score Findings 

Sample 

PLATO Elementary Use Data 

Tests 

Grade 3 Pre- and Post- Scores 

Relating PLATO Elementary Use to Test Scores 

Phone Interviews 

Instructor’s PLATO Elementary Evaluation 

PLATO Elementary Content 

Instructional Design 

Teacher Experience with PLATO Elementary 

Student Experience with PLATO Elementary 

Frequency of Activities 

Open-Ended Questions 

Conclusions & Discussion 

Appendix: Teacher Survey Written Responses 

About the Authors 



.2 

,5 

..5 

..5 

.6 

.7 

.7 

,7 

.8 

.8 

.9 

10 

10 

10 

10 

11 

11 

12 

13 

14 

20 

20 

21 

23 

24 

26 

27 

29 

33 

36 

47 



Apache Junction, AZ 



Page 4 



Apache Junction Unified School District 43 



Introduction 



Apache Junction Unified School District is an organization determined to improve 
education following a multi-year plan. As a part of that plan, they have been working for 
five years to make technology a meaningful part of their educational environment. 
PLATO Elementary 1 was chosen after careful study of several programs, on the basis of 
its service-orientation, overall instructional capabilities, and cost. PLATO Elementary 
was installed in all elementary and middle schools at the beginning of the 1999-2000 
school year. The first year was a learning effort for all, and the second year was more 
rigorous. 

This report is a study of a reading and math remedial summer school program for grades 
1 through 8, conducted in Apache Junction during June of 2001. Classes were held in all 
elementary and middle schools, and with most teachers using PLATO Elementary. This 
program and time period were chosen as the focus of this evaluation. 

The following report first describes Apache Junction District — their interest in 
technology, the process they used in choosing PLATO Elementary, the support they have 
provided for PLATO Elementary, and teacher preparation for technology. This report 
then describes the summer school program in terms of program design, student placement 
in PLATO Elementary, student computer use, and the monitoring of student computer 
use. 

Secondly in this report, the “Data” section describes the various evaluation activities and 
their results as follows: 

Student Use Data and Outcome Findings 

Instructor Interviews — Telephone interviews with three teachers 

Instructor’s PLATO Elementary Evaluation — Written evaluations from three 

schools 

Thirdly, the Conclusions section discusses the findings and relates them whenever 
possible to the overall picture of this PLATO application. The appendix following the 
report gives transcribed comments. 

Program Description 

The District. Apache Junction, Arizona is located 30 miles east of Phoenix and Mesa, 
in the same type of desert landscape. The district has four elementary schools, two middle 
schools, and one high school (which is not a focus of this report). 



1 During the time period of this study, Wasatch Interactive, Inc. was acquired by PLATO Learning, Inc. 
For clarity, all references to Wasatch in this report have been changed to PLATO Primary. -~ecL 
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Elementary Schools: 

Desert Vista Gold Canyon 

Four Peaks Superstition Mountain 

Middle Schools: 

Desert Shadows Thunder Mountain 

Regular elementary school enrollment in the district is approximately 2,450; building 
enrollments range from 550 to 730. The number of teachers ranges from 33 to 35, for an 
average teacher-student ratio of 1 to 17-21. District middle school enrollment is about 
1,300; about 650 for each of the two schools. Like the elementary schools there are 32 to 
35 teachers in each middle school and a ratio of one teacher for every 19 or 20 students. 

In all Apache Junction schools, the majority of students is white. Hispanic students 
comprise 10% to 12% of all but one school; Superstition Mountain Elementary has 18% 
Hispanic students. In all of the schools, approximately 1% of school populations are 
American Indian, Asian, or Black. By locality, one middle school and two elementary 
schools are classified as Urban Fringe of Large City; likewise, one middle school and two 
elementary schools are classified as Rural, inside Standard Metropolitan Area. 

Technology in the District. Apache Junction School District has focused on 
technology in the last few years. Ms. Sharon Smith, Director of Curriculum, reported: 
“We've been recognized as one of six innovative school districts in the nation in our use 
of technology.” For the past five years, the district has been following its multi-year plan 
for implementing technology, which is part of the district’s overall achievement plan. Ms. 
Smith reports that they have worked to lay the foundation for technology use by installing 
cables within schools and fiber optic lines between schools, and hiring technology 
support people. 

Sharon Smith further explains: “Our long-term technology plan includes the development 
of a technology-based student management system, curriculum and instruction presented 
through technology, test item banks linked to the curriculum, and personal wireless input 
devices for students and teachers... The Federal e-rate money and Arizona Technology 
Literacy Challenge grants have been instrumental in helping us implement these 
changes... As an incentive to achieve higher learning, Arizona has adopted Proposition 
301, which provides more money to districts for higher student achievement.” 

As Apache District has pursued its multi-step approach, it has started with one or two 
schools and will finish wiring for the fourth and final elementary school. Four Peaks 
Elementary, this fall. Currently, three elementary schools have a lab and the one depends 
upon clusters of 4-6 classroom computers and rotation schedules. Classrooms typically 
have had from four to seven computers. Teachers who have had inadequate wiring or 
other hardware problems in their classrooms have not been able to use all of the 
computers they have had. Upgrades were required at the beginning of the summer session 
in order to accommodate the Shockwave program. 
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Software Selection. Software selection for the district involved staff, teachers, and 
students. A teacher committee with representation from all elementary and middle 
schools was formed to identify criteria for software selection. Sharon Smith invited 
vendors of interest to set up a system in one of the elementary or middle schools, and 
teachers and students were asked to use them while the selection committee observed. 

The practice users were also asked to evaluate their experience with each program. 

Once the demonstrations were completed for all systems, the 
committee came together to complete ratings of each software 
program. A big issue in evaluating the systems was how service- 
oriented the people who developed and presented and supported 
the systems were. Systems that were developed by teachers, most 
particularly the PLATO Elementary system, were rated most 
highly in this area. The total points were computed for each 
system. Vendors were asked to submit a bid for implementing their 
systems in our district. The committee then gave a proposal to the 
board recommending which system to adopt based upon the point 
assessment and the dollar value of the bids. The PLATO system 
was the highest rated system of those reviewed and they also 
presented the lowest bid. — Sharon Smith 

Support of PLATO. The Curriculum Director is happy with district progress in 
providing technology support to the schools. She says, “We have an extensive system of 
technology support in our district. Each building has a technology support person. The 
district has three floating technology support persons, each of whom is assigned to 
support two buildings. The district also has a technology support group available to all 
buildings.” 

She stated, “If a teacher or administrator has a problem with technology they use an 
online reporting system to report the problem and request service... Every 20 minutes or 
less a technology staff member scans for any problem reports and makes an assignment 
to the school technology person, the floating technology person, or a district support 
person to resolve the problem. This means that within 20 minutes teachers receive a 
response and are told who will be resolving their problem and when the problem will be 
resolved.” 

“Service from PLATO Learning is A+. They get back to me immediately whenever I 
have a question.” 

Teacher Preparation. Director Smith said that more teachers in Apache Junction 
have begun to use technology over the past five years. Competency in technology use is a 
component of their annual teacher evaluation; it is rewarded by being part of the teacher 
merit pay system. Teachers in the district are required to demonstrate proficiency in using 
both the computerized test item bank and the PLATO system. 
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Initial PLATO Elementary training was given during the first year of PLATO Elementary 
implementation — 1999 to 2000. There were three four-hour sessions throughout the year 
for basic training. Teachers who worked in the summer school for 2000 were given a few 
more hours of training. 

A large majority of summer school teachers said that they were adequately trained on 
PLATO Elementary, but in interviews it was noted that only one person could be trained 
at a time, per grade level. That system, combined with a schedule that stretched training 
sessions over several months, led to some frustration for some teachers. Another 
interviewee said that there were also other local training opportunities for those who were 
interested, but that she preferred to learn it on her own. She also said that she likes using 
student reports. “Not everyone likes them; I figured out how to use them on my own so I 
really like them.” 

From the district’s viewpoint, teachers are responsible for integrating the PLATO 
instruction into their overall curriculum throughout the year, and during the school year 
they had grade-level meetings in order to plan for that coordination. Individual teachers 
decide whether students can test out of activities. During the summer school program, 
most teachers focused on providing remediation of specific skills and therefore were less 
concerned about the regular curriculum. They were guided by the program purpose to 
develop deficient skills and demonstrate mastery on relevant PLATO activities. It seems 
that most teachers used PLATO as supplementary to instruction, especially application 
and practice. 

As district policy, teachers are expected to preview all PLATO lessons before assigning 
them to students, so that they are familiar with the instruction presented. During class 
time, teachers assign learning activities, monitor student progress on the activities, and 
troubleshoot any problems. 

Program Design. Apache Junction held summer school from June 4 to June 28, 

2001 for students who were below grade level expectations in reading and math. All 
middle and elementary schools offered the program; approximately 100 students attended 
each elementary school and 75 students attended the middle schools. Students were 
chosen to attend summer school according to scores on district or state criterion- 
referenced tests, other school performance, and teacher recommendations. In each 
elementary, the program was conducted by the principal and one teacher from each grade 
level; about four teachers taught at each middle school. The program was held for four 
weeks, beginning in June. The district required teachers to run student reports, develop 
lesson plans, and submit them regularly. Classes were held for four hours, 7:30 to 1 1 :30 
a.m., Monday through Thursday; the total program time was 1 6 days, or 64 hours. 

Student Placement in PLATO. Every year students are given either the state 
AIMS test or a district version of it that is generated from a computerized test item bank 
and matched to district and state goals and objectives. State tests are given in the third, 
sixth, eighth, and tenth grades; district tests are given in the remaining, or “off-test” 
grades. These criterion-referenced tests determine district success. They are also used for 
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decision-making, such as student selection for the summer school program and 
assignment to PLATO activities. Students were also assigned PLATO activities on the 
basis of teacher recommendations from the former school year and, in some cases, 
according to results of a classroom pre-test. 

Student Computer Use. For the school year, the district has set a minimum level 
of use at 2 !4 hours per week; there is no maximum. During summer school, teacher 
responses indicate that classes had regular blocks of time on PLATO, either in a lab or in 
classrooms. The amount of time was determined in large part by the accessibility of 
computers and lack of hardware/software problems. Most students apparently worked 
alone. One interviewed teacher said that students could not work together because they 
all had to be able to log on [in order to keep records of student work]. 

In phone interviews, two teachers said that they encouraged students by verbally praising 
their efforts and by giving extrinsic rewards; one gave food and one gave little prizes. 

The third said she did not reward behavior, because her class met in a computer lab and 
with another adult to help monitor student work it was apparently not required. 

Computer work presents its own student management challenges. In her lab, one teacher 
commented, the lack of help from someone else was a little frustrating. Classroom 
computer configurations were problematic in more ways. When all students could not use 
the computers at one time, they were divided into groups, and teachers bounced between 
students on computers and those doing various activities. Some made comments on 
evaluation forms about their frustration. In such a setting, students could not be 
monitored as closely in their computer use as teachers would have liked. At the beginning 
of summer school, many students did not know how to log on and off properly, so 
computers froze or would not allow students into the system the following day. Primary 
grade students, in particular, had difficulty because of reading levels required of them, 
especially since they were below grade level. (One first grade teacher did not have 
working sound systems on four of six of her computers and she felt they didn’t learn as 
much as they might have.) Toward the end of the summer school, older students were 
found shifting quickly between screens so that they could send e-mail. 

One teacher benefited during computer use time from the help of a nurse assigned to a 
special-needs student in her room. Once the special-needs student learned how to use the 
mouse “she was good to go”, so the nurse helped other students on the computer. 

One teacher who was interviewed explained that she assigned students to do activities on 
pre-requisite skills before putting them into the regular Math Expeditions program. Many 
of her first grade students, who were being prepared for second grade, had not learned 
such basic concepts as left/right; order words such as first, second, third; and 
above/below. Students were not able to do the math problems because they did not 
understand what was being asked of them. 

Teachers at all grade levels cited difficulty with navigating through lessons; primary 
students had trouble with icons, especially if they were “reading” the colors of the 
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buttons and not the words. One teacher said, “When the ‘You are finished, close the 
window’ message comes, they don’t know how to go to... It doesn’t go automatically to 
the next part; it would be better if it did.” 

When asked about reliability, one interviewee said, “This year it was down for one day 
only. The majority of the time it is okay. They had to upgrade equipment to use 
Shockwave.” However, several written and verbal comments referred to local problems. 
One responder said, “I found that it was difficult to get through all the proficiencies that 
we wanted to teach. We were not able to get to any of the probability or geometry 
concepts at all and could have used a bit more time on fractions. Another week of 
instruction would have been fruitful, or having technology working from the get-go 
might have helped in this area.” 

Monitoring Student Achievement. Teachers monitored their computer-users by 
walking around and observing them and by checking PLATO reports. Many teachers 
liked being able to check whether students were having trouble with mastery tests or not 
spending enough time on activities. Teachers also collected information from other 
classroom activities, and some used benchmark tests produced from the test item bank. 

Teachers who were interviewed said that they used PLATO scores to establish student 
proficiency on learning standards. They graded students for their performance on other 
school work. 

Learning Materials. Those who were interviewed said that they did not assign 
PLATO practice papers, using instead resources of their own and from the district. One 
teacher did not know about the PLATO practice papers. Another one does not use 
paperwork much, preferring to do verbal class exercises. The third grade teacher used 
Scholastic last school year because she taught the one phonics class for her grade level in 
her building; other teachers used Silver Burdett. The math program for her school is an 
objectives-based program produced by teachers because they felt none of the math series 
texts satisfied the state standards. They worked together to make folders and changed 
classes every five weeks. 



Data Analysis 

PLATO Use and Test Score Findings 

During the summer program the typical student used PLATO Elementary programs for a 
little less than six hours over the course of the entire program. Students in fourth grade 
spent somewhat less time on the PLATO programs than students in first, second, or third 
grades. The number of PLATO activities mastered increased from about ten activities in 
the first grade to over 35 activities in the fourth grade, more than tripling from first to 
fourth grade. 
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An analysis of pretest-posttest scores in the third grade summer school class showed a 
significant and large change in both reading and mathematics achievement. The third- 
grade data showed that all of the students scored as at or above the highest third-grade 
pretest score. Additional analysis for first grade, second grade, and fourth grade classes 
showed a generally positive correlation between the level of PLATO Elementary 
program use and posttest student achievement scores. 

Sample. Students in this study used the PLATO Elementary reading and math 
computer programs as part of Apache Junction’s summer school program. The program 
was offered in both elementary and middle schools to help students better master district 
and state learning goals. 

Data from classes of summer school students were used in the study if they had pre-test 
and/or post-test scores and PLATO use data. After polling the elementary and middle 
schools, we identified four classes that had both post achievement scores and PLATO 
system use data. One of these classes had both pretest and posttest data. A total of 71 
students were identified from the records with pre- and post-test data on scales developed 
to measure district and state goals at the appropriate grade level. (Many other classes 
were not used for this part of the evaluation because they had test data but no computer 
record of PLATO Elementary system use, or visa versa.) 



Table 1. Students and Data by Grade Level 



Grade 

Level 


Number of 
Students 


Pretests 


Posttests 


System 

Data 


1 st Grade 


20 


no 


yes 


yes 


2nd Grade 


19 


no 


yes 


yes 


3rd Grade 


17 


yes 


yes 


yes 


4th Grade 


15 


no 


yes 


yes 



PLATO Use Data. The study classes had student-level printouts giving student use 
data for the PLATO Elementary programs. The PLATO system data included: 

• Number of PLATO activities completed 

• Number of activities mastered (activity scores of 80% to 1 00% correct) 

• Average score for all activities completed 

• Total minutes of PLATO activity engaged time 

This data does not include PLATO activities, if any, that students may have studied 
earlier in the school year. Nor does it assure that PLATO activities studied were 
necessarily related only to the pre- and post-test scores available. 

The average number of activities completed by the four classes during the summer school 
program ranged from a low of 15.3 to a high of 39.1 activities; mastered activities ranged 
from 10.3 to 33.5. The average number of activities completed and mastered increased 
from first grade through fourth grade. Activities completed more than doubled from 15 to 
39; activities mastered tripled from 1 0 to over 35. 
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Figure 1. PLATO Primary Activities Completed & Mastered 




The average number of correct answers in PLATO activities completed ranged from 80% 
to 91% correct. Total time engaged in learning on PLATO across the four classes ranged 
from just under five hours to just over six hours. Unlike the other use indicators, engaged 
time did not increase from the first grade to the fourth grade; in fact, it decreased an 
average of an hour and forty-five minutes from third grade to fourth grade. Table 2 shows 
the averages for PLATO use data across the four grades in the study. 



Table 2. PLATO Elementary Use Data by Grade Level 



Grade Level 


Number of PLATO 
Activities 
Completed 


Number of PLATO 
Activities Mastered 


Average PLATO 
Activity Score 


Total Min. on 
PLATO 
Activities 


1 st Grade 


15.3 


10.3 


80% 


374 


2nd Grade 


29.8 


24.6 


88% 


331 


3rd Grade 


30.2 


25.4 


91% 


374 


4th Grade 


39.1 


33.5 


89% 


287 



Tests. Locally developed reading, language arts, and mathematics tests were used in 
the study. The tests were developed at each school from a database of test items linked to 
the district and state learning objectives. The tests were different at each grade level and 
were designed to measure the specific reading, language arts, and mathematics objectives 
which the students needed the most to learn at that grade level. The number of measures 
used depended upon the needs in each school and class. 

Scale scores are reported in terms of percentage of items correct. All four grade levels 
have relatively high percents correct on reading and mathematics posttest scores (see 
Figure 2). The third grade class has substantially higher posttest scores than pretest scores 
(see Table 3). 
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Figure 2. Posttest Reading & Math Scores 
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Table 3. Achievement Test Percent Correct by Grade Level 



Grade Level Pre- Pre-Math Post- Post-Math Post-Lang. 

Reading Reading Arts 



1 st Grade 


(na) 


(na) 


63% 


86% 


(na) 


2nd Grade 


(na) 


(na) 


83% 


80% 


69% 


3rd Grade 


63% 


65% 


89% 


90% 


(na) 


4th Grade 


( na ) 


(na) 


63% 


70% 


(na) 




Grade 3 Pre- and Post- Scores. When the third grade reading and math test 
scores were compared from the pre-test to the post-test, statistically significant gains 
were found for both measures. The mean reading pre-test score was 61 percent correct; 
the mean posttest reading score was 89 percent correct. This is an average reading gain of 
27 percent from pre to post. One common way of judging how meaningful such gain 
scores might be is to use the average standard deviation of the pre- and post-scores as a 
measuring stick. Doing this we find that the mean reading gain of 27 percentage points is 
over two (2.16) standard deviations in size.. 

The mean math pre-test score was 65 percent correct; the mean posttest math score was 
90 percent correct. This is an average math gain of 25 percent from pre to post. The math 
test gain was 27 percentage points and is over three (3.13) standard deviations in size. 

Gains of over two standard deviations are usually considered very large and educationally 
meaningful. These gains reflect the close linkage between the test items and the PLATO 
Elementary activity content. The gain reflects achievement on educationally meaningful 
reading and mathematics content as defined by the district and state objectives. Table 4 
provides descriptive statistics for the pre-, post-, and gains on the third grade scores; 
Figures 4 and 5 shows the pre- and posttest scores for reading and math tests. 
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Table 4. Pre-, Post-, and Gains on 3 rd Grade Reading and Mathematics Scores 



Test 




Mean 


N 


Std. 


Pre-Post 


Post-Pre 


Post-Pre 




Deviation 


Corr. 


Difference 


t test 


Reading 


Pre 

Post 


61% 

89% 


15 

15 


13% 

10% 


.91 


27% 


17.69 


Mathematics 


Pre 

Post 


65% 

90% 


15 

15 


8% 

8% 


.95 


25% 


36.23 



NOTE: Correlation significant atp < .001 ; the t tests are significant atp < .001 



Figure 3. Third Grade Reading Pre-Post Scores 




Figure 4. Third Grade Math Pre-Post Scores 




% Correct 



-Pre Math 



— □— Post Math 



Relating PLATO Use to Test Scores. Many factors typically influence the 
relationship between PLATO use and test scores, such as the way placement and 
individual prescriptions are done, and the degree to which students are allowed to work at 
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their own pace. As a result, the relationship between PLATO use and test scores can 
often be difficult to interpret. To examine the relationship at Apache Junction, the test 
scores within each grade level were divided as nearly as possible into two equal-size 
groups on each of four PLATO use measures. Test scores were then examined for the 
lower- and higher-use groups at each grade level. With a few exceptions, higher levels of 
PLATO use were related to nominally higher achievement test scores. This trend for 
higher scores as related to number of activities completed or mastered is given in Table 5. 
Of the 36 low/high use comparisons listed, 27 (75%) show higher test scores for the high 
use groups. When we remove activity percent correct, 25 of the remaining 27 low/high 
comparisons (93%) show higher test scores for higher use groups. Figures 5 and 6 
illustrate this relationship for activities mastered and engaged time. 



Table 5. Relationship between Posttests and PLATO E 


ementary Use 


Low/High 


Grades 


Reading 


Math 


Lang. Art 


Use Variable 




Lower 


Higher 


Lower Higher 


Lower Higher 


Number of 


1 st Grade 


51% 


75% 


77% 


94% 


(na) 


(na) 


Activities 


2nd Grade 


77% 


89% 


73% 


88% 


59% 


80% 




3rd Grade 


86% 


91% 


86% 


94% 


(na) 


(na) 




4th Grade 


66% 


61% 


69% 


72% 


(na) 


(na) 


Activities 


1st Grade 


55% 


74% 


80% 


94% 


(na) 


(na) 


Mastered 


2nd Grade 


83% 


85% 


78% 


82% 


62% 


75% 




3rd Grade 


85% 


91% 


85% 


94% 


(na) 


(na) 




4th Grade 


68% 


58% 


64% 


78% 


(na) 


(na) 


Activity 


1 st Grade 


62% 


64% 


79% 


93% 


(na) 


(na) 


% Correct 


2nd Grade 


86% 


80% 


83% 


74% 


71% 


63% 




3rd Grade 


89% 


89% 


91% 


90% 


(na) 


(na) 




4th Grade 


64% 


58% 


67% 


77% 


(na) 


(na) 


Engaged 


1st Grade 


60% 


66% 


79% 


92% 


(na) 


(na) 


Time 


2nd Grade 


78% 


87% 


69% 


86% 


58% 


74% 




3rd Grade 


88% 


89% 


89% 


92% 


(na) 


(na) 




4th Grade 


66% 


57% 


39% 


75% 


(na) 


(na) 
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Figure 5. Reading Scores by High/Low Use Groups 




Figure 6. Math Scores by High/Low Use Groups 



? 60% 
o 50% 
^ 40% 




Activities Mastered 



Engaged Time 



□ Math Lower 
■ Math Higher 



When the simple correlations between PLATO Elementary use and test scores were 
examined there were statistically significant correlations for 17 of the 44 pairs of 
variables (39%). Almost all of the significant correlations were in the first or second 
grade. When multiple regression is used to look at the multiple correlations between use 
and test scores there were statistically significant relationships at each grade level 
between PLATO Elementary use and test scores. 
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Table 6. Simple Correlations between Test Scores and PLATO Elementary Use 







Number of Number of 


Average 


Total min. 






PLATO 


PLATO 


PLATO 


on PLATO 


Grade Level 


Test Score 


Statistic 


Elementary Elementary Elementar Elementary 






activities 


activities 


y activity 


activities 






completed 


mastered 


score 




First Grade 


Posttest reading 


Correlation 


0.64 


0.52 


-0.01 


0.44 




Sig. 


0.00 


0.02 


0.97 


0.05 




N 


20 


20 


20 


20 


Posttest math 


Correlation 


0.34 


0.48 


0.43 


0.61 




Sig. 


0.15 


0.03 


0.06 


0.00 




N 


20 


20 


20 


20 


Second 


Posttest reading 


Correlation 


0.45 


0.50 


0.12 


0.68 


Grade 




Sig. 


0.06 


0.03 


0.64 


0.00 




N 


19 


19 


17 


16 


Posttest language arts 


Correlation 


0.53 


0.57 


0.12 


0.69 




Sig. 


0.02 


0.01 


0.65 


0.00 




N 


19 


19 


17 


16 


Posttest math 


Correlation 


0.47 


0.52 


0.17 


0.62 




Sig. 


0.04 


0.02 


0.50 


0.01 




N 


19 


19 


17 


16 


Third Grade 


Pretest reading 


Correlation 


-0.08 


-0.05 


0.06 


-0.16 






Sig. 


0.77 


0.86 


0.83 


0.55 




i _ 


N 


16 


16 


16 


16 


Posttest reading 


Correlation 


0.27 


0.37 


0.14 


0.10 




Sig. 


0.33 


0.18 


0.61 


0.72 




N 


15 


15 


15 


15 




Pretest math 


Correlation 


0.34 


0.43 


0.10 


0.06 






Sig. 


0.20 


0.10 


0.72 


0.83 




i . 


N 


16 


16 


16 


16 


Posttest math 


Correlation 


0.42 


0.47 


0.01 


0.22 




Sig. 


0.12 


0.08 


0.97 


0.44 




N 


15 


15 


15 


15 


Fourth Grade Posttest reading 


Correlation 


-0.41 


-0.43 


-0.16 


-0.35 




Sig. 


0.13 


0.11 


0.60 


0.24 




N 


15 


15 


13 


13 


Posttest math 


Correlation 


0.11 


0.18 


0.58 


0.18 




Sig. 


0.69 


0.53 


0.04 


0.57 




N 


15 


15 


13 


13 



0 
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Table 7. Multiple Regression Correlations between PLATO Elementary Use and 
Test Scores 











Standardize 






Dependent 

Variable 


Model Elements 


Unstandardized 

Coefficients 


d 

Coefficients 


t 


Sig. 






B 


Std. Error 


Beta 






First Grade | 


Posttest 


(Constant) 


0.058 


0.222 




0.260 


0.798 


Reading 
R 2 = .275 


Number of PLATO Elementary 
activities mastered 


0.056 


0.021 


0.524 


2.613 


0.018 


Posttest 


(Constant) 


-0.985 


0.388 




-2.543 


0.021 


Math 
R 2 = .645 


Average PLATO Elementary 
activity score 
Total minutes engaged in 
PLATO Elementary activities 


1.897 

0.001 


0.464 

0.000 


0.569 

0.715 


4.087 

5.139 


0.001 

0.000 


Second Grade i 


Posttest 


(Constant) 


-0.579 


0.383 




-1.514 


0.154 


Reading 
R 2 = .658 


Average PLATO Elementary 
activity score 
Total minutes engaged in 
PLATO Elementary activities 


0.994 

0.002 


0.370 

0.000 


0.481 

0.888 


2.685 

4.959 


0.019 

0.000 


Posttest 


(Constant) 


-1.894 


0.786 




-2.410 


0.031 


Math 
R 2 = .596 


Average PLATO Elementary 
activity score 
Total minutes engaged in 
PLATO Elementary activities 


1.959 

0.003 


0.760 

0.001 


0.502 

0.836 


2.578 

4.296 


0.023 

0.001 


Post Lang. 
Art 

R 2 = .653 


(Constant) 

Average PLATO Elementary 
activity score 
Total minutes engaged in 
PLATO Elementary activities 


-1.752 

1.691 

0.003 


0.672 

0.650 

0.001 


0.469 

0.886 


-2.606 

2.601 

4.911 


0.022 

0.022 

0.000 


i Third Grade 


Posttest 


(Constant) 


0.445 


0.049 




9.103 


0.000 


Reading 
R 2 = .876 


Pretest reading score 
Number of PLATO Elementary 
activities mastered 


0.639 

0.002 


0.075 

0.001 


0.873 

0.225 


8.481 

2.191 


0.000 

0.049 


Posttest 


(Constant) 


-0.444 


0.137 




-3.243 


0.009 


Math 


Pretest math score 


1.040 


0.046 


1.067 


22.442 


0.000 


R 2 = .986 


Number of PLATO Elementary 
activities completed 


0.013 


0.002 


2.518 


7.674 


0.000 




Number of PLATO Elementary 
activities mastered 


-0.015 


0.002 


-2.295 


-7.652 


0.000 




Average PLATO Elementary 
activity score 


0.736 


0.144 


0.410 


5.118 


0.000 


| Fourth Grade 




(Constant) 


-0.711 


0.605 




-1.175 


0.265 
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Posttest (Constant) -0.711 0.605 -1.175 

Math 

R 2 = .337 

Average PLATO Elementary 

activity score 1.611 0.681 0.581 2.365 
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Phone Interviews 

During the fall of 2001, three elementary school teachers were interviewed by phone, 
using the PLATO Instructor Interview Questions. The teachers — one third grade and two 
first grade teachers — were from different schools in the district. The responses to these 
interviews were used in the program description at the beginning of this report and other 
sections to bring some depth to other comments or data. There is one issue from the 
interviews that has not previously been discussed and that is presented here. 

Was there a regular time within the sequence of a lesson or unit in which your students 
experienced their PLATO activities? 

All three teachers used PLATO as a supplement to their classroom instruction. These 
respondents did not seem to think in terms of parts of a lesson beyond introductory 
instruction, application/practice, review, and assessment. When discussing PLATO they 
framed their use pattern in terms of availability of computers. Especially for the summer 
school program, all of those who were interviewed spoke about meeting goals for 
remedial instruction; learning gaps guided what they did with PLATO, as well as the 
availability of computers. One trip a week to a lab is, by necessity, an occasional 
experience. Having a few computers in classrooms requires use throughout the school 
“day”, and therefore is more loosely tied to a teacher’s lesson. 

One teacher said, “We have five computers in my classroom to use on a rotating basis. 
We also went to the computer lab once a week. I have 20 to 30 students in my class 
(different grade levels last year and this) and it takes a long time to get them all through 
the classroom computers. I had students use computers during reading group time, not 
during reading instruction time.” 

Another teacher said that she uses PLATO for practice and reinforcement, that students 
should be taught concepts first and then practice the concepts on the computer. “I do a lot 
[with computers] during silent reading. Also, those who get done early [with classroom 
work] get in and work where they left off [on the computer]. 1 don’t assign the next thing 
until they are done with one assignment [they are working on].” 



Instructor’s Evaluation 

In order to gather information from several teachers who are using PLATO Elementary 
software in the district, the Instructor’s PLATO Evaluation survey was distributed to 
three schools. (Surveys from two other schools were returned to the evaluators too late 
for inclusion in the report, but the content of the late surveys are consistent with the 
findings reported here.) The survey asked 41 questions about teachers’ use of PLATO, 
divided into three sections. Part I had 28 questions about the PLATO Elementary 
products and their use. Teachers were given a five-point scale of answers, from “Strongly 
Agree” to “Strongly Disagree.” Part II of the survey asked how often teachers gave 
instructions to students regarding the use of the PLATO system; teachers answered on a 
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six-point rating scale, from 5 (daily) to 0 (never). Part III of the survey posed seven open- 
ended questions, to be answered in writing. 

The faculty members of two middle schools and one elementary school completed the 
Instructor’s Evaluation. The schools represented were Thunder Mountain Middle School, 
19 usable forms returned; Desert Shadows Middle School, 30 usable forms returned; and 
Four Peaks Elementary School, 24 usable forms returned. The total number analyzed for 
all three schools was 74. A few respondents wrote a note that they were new in using the 
program but they nevertheless answered at least Part I of the questions. 

Results of this survey point out the positive aspects of PLATO Elementary for Apache 
Junction teachers were primarily screen design, lesson content, and teacher training. 
Results also disclose five problematic issues that are indicated elsewhere: Software bugs 
and problems 2 , students being confused or trapped in the program, and teacher 
frustration, especially in terms of lack of computer access and the resulting complexity of 
running student “groups”. 

Comments from the other sources hint that, generally speaking, PLATO Elementary 
rankings were good when hardware was available and working, PLATO Elementary 
software worked well, and PLATO Elementary topics were applicable and acceptable to 
the teacher’s subject area. There was also some feeling displayed in the other data 
sources that computers in general were a different matter than specific software or 
individual school equipment settings. 

PLATO Elementary Content. The first group of questions discussed here asked 
about the content of PLATO Elementary lessons and their alignment with state, district, 
and teacher objectives. From one-half to three-fourths of all respondents agreed that 
content was good. Most positive responses came for content being current (Q. 5), content 
being good for the topics (Q. 1), and course objectives aligning with the teacher’s own 
(Q. 2). There seemed to be concern over PLATO Elementary alignment with the standard 
final test (Q. 3) and content being free from errors (Q. 4). The Neutral category included 
18% to 29% of the responses. Elementary teachers were slightly more negative than were 
middle school teachers on most of the items in this cluster. A few first grade teachers 
wrote comments about issues related to beginning readers who could not benefit from 
PLATO Elementary lessons or who struggled with the structure of the system, 
particularly in this remedial program. 



2 Apparent bugs can be caused by the software or the configuration of the local hardware, operating system 
or network. No attempt was made in this evaluation to isolate root causes of apparent bugs. Software bugs 
and errors are corrected in regular maintenance releases, and technical support assists clients with 
hardware, software and networking issues, -ed. 
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Figure 7. PLATO Content 




Number of Teachers 

[□Strongly Agree □ Agree □ Neutral □ Disagree □ Strongly Disagree) 



Figure 8. PLATO Content Mean Ratings 



5. Content was generally up- 
to-date. 

1 . PLATO content for these 
topics is good. 

2. PLATO course 
objectives aligned to my own. 

8. There was adequate 
depth in exercises and tests. 

7. Students generally 
understood the explanations. 

4. Content seemed generally 
free of errors. 

3. PLATO content aligns 
with our standard final test. 




1.0 



2.0 



3.0 



4.0 



5.0 



1=Strongly Disagree, 2=Disagree, 3=Neutral, 4= Agree, 5=Strongly Agree 




23 



Apache Junction , AZ 



Page 22 



BESTCOPY AVAILABLE 



Instructional Design. Approximately three-fourths of the respondents were positive 
about PLATO Elementary software. Respondents were most positive about the graphics 
(Q. 14, 84%) color (Q. 13, 84%), and screens (Q. 15, 82%). They were also positive 
about the style of instruction being consistent (Q. 6, 74%), alignment of lesson parts with 
Instructor Guides (Q. 9, 72%), consistent use of keystrokes and display style (Q. 12, 
70%), and tutorials involving students through interactive style (Q. 1 0, 68%). 

Figure 9. Instructional Design 

14. Graphics were used appropriately. 



12. PLATO used consistent keystrokes, display style. 
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Figure 10. Instructional Design Mean Ratings 
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Fewer positive ratings were given when educators were asked whether software was 
generally free of bugs and errors (Q. 1 1). Only one in three respondents (32%) agreed 
that PLATO Elementary satisfied that criterion; this was the most negative score in the 
survey. Fully one-half of the respondents (50%) disagreed that the software was free of 
bugs or errors, and one in six (18%) were “neutral”. Elementary teachers were less 
positive about tutorials involving students; they were a little more positive about the 
content being free from errors. 

Teacher Experience with PLATO Elementary. Three out of four respondents 
(Q. 27, 73%) said that they had been adequately trained to use PLATO; elementary 
teachers were a little more positive about the training. One-half of the respondents were 
interested in more training (Q. 28, 51%), with middle school teachers being more 
interested. 

Six out of ten teachers (62%) indicated that they could make assignments on the system 
(Q. 1 8). Almost one-half of the respondents agreed that they were able to use student 
progress reports (Q. 16, 46%) and that they were able to relate PLATO Elementary to 
classroom activities (Q. 20, 46%). Most of the remaining ratings for these two items were 
“neutral” (39% and 38%, respectively). Phone interviews indicated that student progress 
reports were used daily by some teachers to monitor student work, but they were used 
only occasionally by those who relied more upon their own judgment of activities for 
assigning activities. Low ratings for relating PLATO Elementary with activities, are 
probably tied somewhat to the atypical situation of summer school. One teacher in a 
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phone interview explained that she did not try to tie PLATO Elementary to her usual 
curriculum because her focus was working to mastery level on specific skills. Also, many 
teachers may have used classroom activities for other purposes. 

Approximately four out of ten teachers said that they enjoyed working with PLATO 
Elementary (Q. 25, 43%), that PLATO Elementary was useful in teaching (Q. 26, 41%), 
and that computer work is productive (Q. 24, 38%). The questions that have been 
grouped for this cluster all received a large percentage of “Neutral” rankings — from 20% 
to 39%. In fact, some of the rankings were fairly evenly distributed across all categories. 
Rankings of “Disagree” or “Strongly Disagree” ranged from 7% to 34% of the responses. 
Most negative of the items was Number 24, which stated that computer work is 
productive and not frustrating. This was one of the five most negative responses on the 
survey. 



Figure 11. Teacher Experience with PLATO 
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24. Computer work is productive, not frustrating 
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Figure 12. Teacher Experience Mean Ratings 
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Student Experience with PLATO Elementary. When asked whether students 
responded well to PLATO Elementary, one-half of the teachers agreed (Q. 23, 49%) and 
one in six disagreed (1 8%). Likewise, when asked whether students in general responded 
well to PLATO Elementary, one-half of the teachers agreed (Q. 21, 49%) and one in six 
disagreed (21%). Elementary teachers were slightly more negative on these two items, no 
doubt because they had more trouble in general with students getting in and out of the 
program and understanding the instructions. From phone interviews, it seems that 
students worked alone on the computers, and so there was no peer assistance. 

Only one in four teachers said that they could do tutoring while students used PLATO 
Elementary (Q. 17, 23%); four in ten disagreed (39%) and a large proportion was neutral 
(38%). These results are consistent with the description of computer availability and 
software problems. Teachers were busy helping students with system use, while 
managing other instructional groups. Linked with this issue is that of some students’ 
difficulty in using the system. In response to the statement that students are seldom 
confused or trapped, only one in five agreed (Q. 22, 19%). One-half of the respondents 
(49%) disagreed, thus indicating that student use was a serious problem. One-third of the 
teachers were “neutral” on the issue (32%), because they either did not use the program 
or use it much, or because they had a mixture of experiences, with students becoming 
more capable toward the end of summer school. 
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Figure 13. Student Experience with PLATO 
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Figure 14. Student Experience Mean Ratings 
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Frequency Of Activities. The second section of the Instructor’s PLATO Evaluation 
asked for ratings of how frequently teachers explained six issues to their students. The 
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areas of explanation were: (Q. 29) prerequisites for success with PLATO; (Q. 30) 
personal experiences related to PLATO assignments; (Q. 31) learning objectives; (Q. 32) 
how PLATO fits into course goals; (Q. 33) incentives for doing well with PLATO; and 
(Q. 34) procedures for getting help on PLATO. The most commonly explained issues 
will be discussed first. 

Across all of the categories, the averages of all possible ratings are fairly similar, ranging 
from 4.1 to 3.5 on a scale of 1 to 6. The patterns across categories were not strikingly 
different, either, including fairly even distribution of answers across the time choices: 
approximately one-third were performed frequently, occasionally, and seldom. 

The highest overall average came from respondents who said that they explained 
procedures for getting help with PLATO Elementary (4.1); these explanations were most 
likely to be given daily or almost daily (43%). Furthermore, only 12% said that they 
never gave such an explanation. These responses indicate some level of concern or 
problem that would motivate relatively consistent behavior. 

The twin issue of importance for teachers was describing learning objectives to students 
(3.7). Explanations were given daily or almost daily by almost half of the respondents 
(45%). This is logical in light of the program being tightly focus on specific learning 
objectives. In regular classrooms, this might not be so high. 

Approximately one out of three respondents gave “daily or almost daily” instructions for 
the remaining categories. 



Figure 15. Frequency of PLATO Activities 
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Figure 16. Mean of PLATO Activities 




1=Never, 2=0nce, 3=Beg Inning, 4=0ccasionai(y, 5= Most, 6=Each 



Figure 17. Mean Elementary & Middle School Teacher Responses 




Open-Ended Questions. Instructors were asked seven questions at the end of the 
survey, with space provided for their written responses. Responses were tabulated and 
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collapsed into general categories for a condensed view of issues that teachers mentioned. 
The categories that were generated and the numbers of teachers referring to them appear 
below. The total number of responses for each question does not indicate the number of 
respondents; some responses addressed more than one issue. (Responses were transcribed 
and are presented in the appendix.) 

1 . What do you like best about teaching with the PLATO computer? 

A majority of teachers wrote that what they liked best about PLATO Elementary was that 
it was motivating, great for application and practice, self-paced, and a different way to 
reach learners. 



Motivating to students, interesting 1 1 

Great for application/practice 1 0 

Self-paced 9 

Different way to reach learners 8 

Codes; alignment with proficiencies 4 

Student reports; ability to track 4 

Good for diagnosis and remediation 3 

Good for reviews 3 

Don’t like it 1 1 

Don’t use; my subject not included 4 

Hard to use, not enough time/help 6 

Haven’t used 1 



2. What do you like least about teaching with the PLATO computer? 

Technical problems were most frequently mentioned by respondents as being what they 
liked least. Student difficulty and the time required to use the program, taken together, 
were also notable. 



Technical Problems 26 

Content Issues 13 

Access 3 12 

Student Difficulty 12 

Time Consuming 1 1 



3. Was there a regular time within the sequence of a lesson or unit in which your students 
experienced their PLATO activities?... 

Activity, Application, Practice 19 



3 Given the student:computer ratio at Apache Junction, this is probably best interpreted as related to a need 
for further training in how to integrate computers into daily classroom activities, rather than as a need for 
more hardware. PLATO Elementary software is designed to support a wide range of learning activities 
with a range of studentxomputer ratios. 
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Time, misc. 
Review 

Complete Lesson 

Assessment 

No 



10 

7 

6 

4 

5 



4. Describe any strategies you employed to determine whether or not the PLATO 
activities assigned to each student were the most appropriate for ensuring their 
success in your class. 

The largest category of answers, all from middle school teachers, said that trying lessons 
themselves was their best way of determining whether activities were appropriate. A 
sprinkling of middle school instructors (10) listed several other options. Nine elementary 
staff comments were: checking codes (3), monitoring outcomes (2), the use of peers (2), 
and using a recommended program (1), and three said that they were confused and did 
not have a strategy. 

Tried them myself 1 5 

Codes 5 

Monitored outcomes 4 

Helped, asked students 4 

Use of peers 2 

Used recommended program 1 

Asked Students 1 

None, confused 3 

5. How would you change the PLATO Elementary lessons? 

Two elementary teachers said they would build in assessments or codes for necessary 
pre-requisite skills for each lesson; sometimes students do not have all of the entry skills 
to do the lessons, even though they do have some. One middle school instructor 
suggested making a PLATO course to explain how to use the system; three others 
suggested making the program easier to use. The “telephone” was mentioned in particular 
by one teacher as being something her students had trouble with. 

Add to, improve content 9 

Good as they are 8 

More direct, objective-specific, mature 6 
Easier to work with 4 

Greater computer access 2 

Not on Internet 1 

Want to use AA also 1 

In phone interviews, teachers said the following: 



Apache Junction, AZ 



Page 31 



“It would be nice if the content were a little more specific for the students I teach. Some 
of the programs are too far advanced, too hard for my students; some of the modules are 
too simple. The program needs more content this is specifically tied to my grade level [1 st 
grade]. “ 

“In the math program, in the lessons on decimals and fractions, the program does not give 
a star for completed lessons when the student has done problems correctly. She said the 
company is fixing it.” 

‘I like how it talks to them.” 

“Improve the log in. I understand the PLATO program, but it asks for day-month-year, 
backwards from what we usually do in this country. It asks for certain parts of the 
[child’s] name. Kids get lost, it is too long and complicated. A simple number would be 
best. As it is now, they log in from the desktop, then click on the PLATO two times. It is 
easier to do it myself. I have trained two kids to do it. One is a student who was retained; 
it is good for that student.” 

6. What suggestions do you have to improve the way you use the PLATO Elementary 
system? 



Working computers 8 

Provide more time 2 

Learn to use it more 2 

Align PLATO Elementary with class 1 

Delete old assignments 1 

None 1 

Miscellaneous 5 



7. What other comments or suggestions do you have on the PLATO Elementary system or 
this course? 

There were few comments on this last item. Of the responses, most regarded students 
having difficulty with content in some way and needing an easier way to use the program, 
particularly logging in and out. 



Student problems with content 7 

Easier use, incl. log-in and -out 6 

Technical problems, incl. access 5 

More, better training; practice 4 

Access to working computers 3 

Like program, training 4 

Don’t like program 1 



The following few comments gave particular detail of potential interest: 
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“When assigning lessons by the curriculum codes, sometimes the PLATO Elementary 
activities did not match up complete with the objective. For example, the ‘setting’ of a 
story has only one question on the setting, but a whole activity on writing a script for 

TV” 

“Quality — No good in language arts; consistent — yes, but not good.” 

“Kids I see are generally guessing on answers — not thinking about them. It becomes a 
‘guessing game’ and I’m not sure how well they’re using their time. I have more one-on- 
one time with them because I work with just PLATO kids. In order for the projects to 
really be effective I stay right with them. [She works in the library.]” 

“Are there lessons for intermediate kids that are at a lower reading level? Most 
intermediate objectives have grade-level vocabulary which my lower kids can’t read 
independently.” 

One respondent said, “I tried to use the computers with these students, but ran into 
repeated problems getting them logged on because they were not DSMS students. Prior to 
summer school all the 5th grade names were added to the DSMS list. The trouble 
occurred when trying to get them logged on. They were still listed at their other school. 
Our tech support person spent many hours working on the problem. My groups used 
PLATO when they could get on. The program was a good resource for reinforcement 
when it was working. Program difficulties included: Shockwave, running clock, files not 
found, deleted icons, etc.” 

“We attempted to use [PLATO Elementary and PLATO Secondary] to help us teach and 
reinforce the concepts taught but ran into much difficulty there. We had trouble with 
Shockwave, files not found, hour glass problems and log on problems, to name a few. 

The technology group worked with us and we whittled our way through some of the 
major problems. Because of the technological difficulties, we were forced to rely on 
traditional teaching methods for the most part. I was able to use the CPS program the last 
couple of days for some trivia games, just to get the students' reaction to using the 
transformers. This was very successful and positively received by the students.” 

One written comment referred to middle school students hiding the icons and exiting 
improperly, he thought, on purpose. “The administration was very supportive and those 
students who were excessively tardy or missed 3 days of instruction were removed. There 
were literally no behavior problems that were not handled swiftly and fairly.” 



Conclusions & Discussion 



Selection of PLATO — PLATO Elementary was chosen because it received the highest 
number of points on their committee’s evaluation and because it was lowest in cost. An 
important issue in the evaluation was its service orientation in these areas: (1) the 
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instructional design, which emphasizes interdisciplinary, high-level thinking skills (2) the 
sales representative, and (3) the support system once the program was in place. 

Teacher Training — Three out of four teachers who responded said that their training was 
adequate. Nevertheless, it was found that in spite of providing 4-12 hours of training to 
every teacher 4 , training does not appear to be completely effective, since teachers do not 
spend much time training their peers in such roll-outs. In addition, various comments and 
ratings alluded to difficulty in using the system. One-half of these teachers said that they 
were interested in further training. 

Motivation in Seeking Software — State standards and criterion-referenced tests are 
driving instruction in this state (and others). The major goal for educators is satisfying 
these requirements. Instructional time is an underlying element of success, and 
interventions that are time-consuming in or out of class can cause a great deal of 
frustration. As never before, teachers are interested not only in ease of use, but also in 
seamless, straight-forward programs that help them be efficient. 

Student Use and Test Results — The preliminary finding in this evaluation is that 
participation in the summer school program, including participation in the PLATO 
Elementary program, was positively correlated with higher student achievement. An 
analysis of pretest-posttest scores in the third grade summer school class showed a 
significant and large change in both reading and mathematics achievement. Additional 
analysis for first grade, second grade, and fourth grade classes showed a generally 
positive correlation between the level of PLATO Elementary program use and posttest 
student achievement scores. Given the limited sample size, this finding will need to be 
replicated and extended through other studies. 

Use of PLATO Elementary — Most teachers did not use PLATO Elementary as their 
major means of instruction; rather, they used it for application, practice, or review. Those 
who liked the program (43%) saw this as a great need and valued its use in that role, in 
spite of its difficulties. Those who did not like it had special issues with the following 
concerns: 

a. Teachers trust their own explanations and methods more than software. 

b. Low-achieving children (in first grade, particularly) are language, experience, 
and vocabulary deficient and may not be ready for grade level instruction. 
Also, they may have a mixture of abilities, such as adequate reading abilities 
and deficient math abilities, such that a lesson is difficult to master. 

c. Content did not align with state and local standards. 

d. Some content was unequally addressed. Some of the objectives teachers 
wanted to teach were given light coverage in the PLATO Elementary 
modules, while other less important matters were covered at length. 



4 Some teachers reported that only one teacher per grade was trained. Since this is at variance with the 
facts, there may be a perception issue surrounding training. 
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Strengths of PLATO Elementary — The program was appreciated for its presentation 
quality and consistency. Teachers also liked the variety the software offered to the overall 
program and the additional student practice. 

Problems with Using PLATO Elementary — Below are the major issues that were 
represented throughout the data. 

a. Computer access, especially in light of district demands for amount of student 
computer use requirements, was a major cause of much of the frustration that 
educators have experienced throughout the two years they have used it. The 
district’s success cannot be total when computer setups produce complex 
classroom management arrangements. Frustrations over high demands in the 
face of lack of opportunity no doubt colored the entire experience for many 
teachers. 

b. Bugs and software glitches were a major problem for half of the respondents. 
As one teacher wrote, “I think overall, the kids benefit from the program. It is 
just that sometimes technology doesn’t always work for us.” 

c. Getting into the program, navigating through it, and exiting were great 
problems for many students. Perhaps these problems were exacerbated by the 
remedial status of the students, but these comments seemed to apply to teacher 
experience in general . 5 

Student Experience — It seems from the data that there is little extrinsic encouragement 
for students in some classrooms for using PLATO. Rewards should be built into the 
program by way of outstanding ease-of-use and frequency of positive feedback. 

Overall, it seems that equipment availability limits the effective use of any software. The 
particular problems PLATO Elementary has presented, however, make it apparent that 
changes must be made if the program is to develop a track record of offering true 
advantages in the classroom. 



5 It seems likely that these issues are related to a combination of interface design in the management 
system, and administrative procedures for registering students. Since the time period of this study, the 
PLATO Primary courseware has been incorporated under a different management system, PLATO 
Pathways, which is used for all PLATO courseware, -ed. 
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Appendix; Teacher Survey Written Responses 



The following section presents the responses to open-ended items in Part III of the 
Instructor’s PLATO Evaluation. In general, answers were transcribed as they appeared. 
However, for greater ease in reading, spelling was corrected and in a few instances one 
word was added that apparently had been omitted in error. (Brackets were not used for 
these added words in order to facilitate ease of reading.) The responses below are labeled 
with the number assigned to the returned survey; skipped numbers indicate that 
respondents had not answered those items. 

1 . What do you like best about teaching with the PLATO Elementary computer? 
MIDDLE SCHOOL #1 

01 I like that I have a different way to try to reach all my learners more effectively. 

02 I don’t like it. 

03 I don’t use it because I teach computer apps. 

04 The student reports. 

05 The ability to track student progress. 

06 It is self-motivating for the students. It covers a lot of material and is great for 
applying skills learned in class. 

07 Haven’t used long enough to know yet. 

08 I do not like using PLATO Elementary. 

09 In language I find that using PLATO Elementary was more difficult. Our skills in 
language focus mainly on writing. I do a lot of my grammar lessons with writing 
assignments. I then do my lessons. 

1 0 I don’t use it because of the subject I teach. 

11 It is too confusing to use and we do not have complete labs or teachers in the lab to 
help students. 

12 The students already know how to use them from elementary school. 

13 I prefer to use PLATO! Easier to use. 

14 Do not enjoy using PLATO Elementary. Not American history; reading is 
elementary. 

16 The interaction of computers and students — it is a nice change of pace from just 
classroom settings. 

17 It gives me another tool. 

1 8 Self-paced for every student; great reports. 

MIDDLE SCHOOL #2 
Veteran Teachers 

19 Diagnose and remediate. 

20 Assists students who need additional help. 

22 Kids work. 

23 The codes corresponding. 

24 Reinforces basic math skills. 
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25 Specific to Aims codes. 

26 Interesting activities. 

27 Does not correspond or supplement band instruction. 

28 Extra set of hands. 

29 Students who are completed or are behind can use PLATO Elementary to either move 
on or catch up. 

30 Nothing. 

31 Individualized practice. 

32 Challenging higher students while offering tutorials for lower students. 

33 1 didn’t. For English (which 1 taught the previous two years) PLATO Elementary was 
unclear for the students and did not provide user-friendly tutorials. In fact, PLATO 
Elementary is confusing to students. I used PLATO only for English as a result. 

34 Students work at own pace. 

36 The connection to proficiencies. 

37 It gives students a break from the lecture method of teaching. 

38 Knowing that my students are learning in a more “well rounded” fashion. 

39 Good for reviewing prior content. 

New Teachers 

40 It adds to instruction. 

4 1 It meets the needs of students. 

42 I can individualize instruction. 

43 Interactive and interesting for the students. Self-directed learning. 

44 I am able to provide and monitor feedback to the individual students who need it the 
most. 

46 Not much right now — I’m still confused and feel like I just throw an assignment at 
the kids. They do not have adequate time. 

47 Have not taught with PLATO Elementary yet. 

48 The fact it is a good tool for review. 

49 I don’t know — I just received first training session yesterday. 

ELEMENTARY SCHOOL 

50 Starting to be matched to state standards. The actual PLATO Elementary program is 
great! 

51 I like the additional chance to practice reading and math concepts. 

52 The children are learning and having fun doing it. 
new53 Another way of teaching. 

54 Kids are motivated. 

55 This is a great way to review skills and practice. 

56 It’s very motivating for the students. 

57 Reinforces skills. 

59 Another way to practice. 

61 At this point I find it frustrating! 

62 It’s another way of teaching and more time on task. 

63 The students are able to work with it independently. 

64 I like how we can reinforce our teachings with PLATO Elementary. 

65 It gives students extra time to work on skills. 
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66 Individualized instruction for children. 

70 Reinforce basic skills. 

2. What do you like least about teaching with the PLATO Elementary computer? 
MIDDLE SCHOOL #1 

01 Last year it was down a lot. I was frustrated quite often. Students would delete the 
icon and I didn’t know how to get it back right then and there. I would have to put a 
ticket in and wait. [Note on Question 27, Part I.]: The PLATO Elementary system was 
down for most of my training. 

02 My students were confused. I didn’t feel like it matched my objectives in the 
classroom. 

05 Minor glitches. 

06 It is often inaccessible. I can’t find adequate time to allow students on the computers. 

08 I feel it is confusing to the students. Each of my students have expressed how easy it 
is, and that they have not learned from it. 

09 I don’t teach like that. I’m not as comfortable using tutorial. 

1 1 Hard to schedule; too time consuming; uninspiring. 

12 The wait when they log off wrong. 

14 Students find it too easy or too confusing — no in between. Irrelevant. 

1 6 Sometimes computers aren’t working right. 

17 Problems with computers — and getting access to computers. Note — I’m trying to use 
it in both social studies and reading. I’m having some problems. 

18 Has proved unreliable. 

MIDDLE SCHOOL #2 
Veteran Teachers 

1 9 At times system is quirky in setting up assignments. 

20 It is too easy for a student to log off improperly and then be locked out for the day. 
Moving through some of the assignments was confusing. 

22 Computer problems. 

23 I prefer less “fluff’. 

24 Shockwave made it impossible to use PLATO Elementary for the first two weeks of 
school. 

25 Computer problems. 

26 The integrated activities don’t allow my students to focus on specific skills. 

29 A lot of students have too many problems. Some codes are missing. 

30 Not enough computers. Computers freeze up. Not enough time to view assignments 
myself to explain them. Too difficult to group and do class instruction. 

3 1 Computers freeze up. 

32 (1) Freezing screens. (2) Changing from activities after completing is confusing for 
students — stop, menu . 

33 The fact that I had PLATO as an alternate tool. PLATO Elementary is very weak in 
its English section. 



Apache Junction, AZ 



Page 38 



34 Time spent waiting for screens. Lack of direction for kids’ understanding. Icons. 

35 Takes up too much of my time. Many students just not motivated to work alone — 
especially low readers. 

36 Shockwave and technological errors. 

37 Not convinced it is the ideal way to support my science class. 

38 Confusion and glitches. 

39 Computer glitches. 

New Teachers 

40 A lot of teacher management. 

42 Many of the objectives I need to teach only have one small lesson. 

43 Benefits my students. 

44 When the program of other technical problems become a distraction or obstacle, 
especially with large class sizes. 

46 The kids know more than I do. I’m frustrated and am having trouble with reports. 

47 Have not taught with PLATO Elementary yet. 

48 The time it takes to schedule time. 

49 Does not work consistently — electrical problems — error messages consistently. 
ELEMENTARY SCHOOL 

50 System bumps students out — students are not able to work independently for a set 
amount of time. 

5 1 The difficulty of children logging on (not enough assistance during the children’s 
computer time). The computers freezing up. 

52 PLATO ELEMENTARY is wonderful. However, the children are either locked out 
of the system (we have to continually put tickets in for system problems), or the system is 
down. 

53 Computer freezing up (building problem). 

54 A child may be on a certain reading level, but the math is too difficult (on projects). 

55 It takes me a lot of time to assign assignments. 

56 The bugs. — It’s difficult to manage kids on computers plus the others in the room. 
The kids on computers often need much more attention. 

57 Not having time to work with the students — a lab would provide this. 

58 Teachers need more time. ..[See number 7] 

59 System bugs and down time. 

60 Not enough TIME. Not enough computers. 

61 [Written on Part I question #19, My students were scheduled... as much time. ..as 
needed]: Not enough computers. 

62 Not always available — Freezes and shuts down in the middle of programs. 

63 Much of the vocabulary is difficult for intermediate special education students. 

64 Not enough time or computers. 

66 The “power” issue at our school; we cannot get students on for enough time each 
week. 

68 [Written on Part I question #24, 1 find working with the computer is generally a 
productive, rather than frustrating, experience.] Time. 

70 Not enough TIME. 
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3. Was there a regular time within the sequence of a lesson or unit in which your 
students experienced their PLATO Elementary activities?... 

MIDDLE SCHOOL #1 

01 I used it to enhance a unit. For example: the Persuasive Writing unit for Language. I 
used it for application for math. 

02 No. 

05 I use PLATO Elementary to teach a complete lesson once a week. 

06 Practice, review, application, and assessment. 

08 I do not use PLATO Elementary to teach my class. If there is time left when they are 
finished with their assignment, they can get on. 

09 It was used as an assessment. If they had a problem then we discussed it. I used 
mainly in activity period. 

12 Practice and review. 

16 We will use it during the activity. 

17 I use it for research and current events — periodically — each quarter. 

MIDDLE SCHOOL #2 
Veteran Teachers 

19 They use PLATO Elementary for all of the above mentioned areas. 

2 1 Labs. 

23 Assessment tool. 

24 Used during mindset, after lecture. 

25 An assessment tool. 

26 I use PLATO Elementary mainly for application once the concepts are learned. 

29 Used during the practice/review. 

3 1 They used only for practice and review. 

32 Activity/information phase in the class, not lab. 

33 Yes — We visited the lab for 30-40 minutes every Thursday in each one of my classes. 

34 PLATO Elementary was used after a pre-test, then students were given assignments 
in weak areas. 

35 Various times. 

36 Not this year; computers not available. 

37 No. 

38 Students used PLATO Elementary at various times during the unit. 

39 Review only at algebra level. 

New Teachers 

40 During mindset. 

42 Yes, I use PLATO Elementary for practice. 

43 Activity phase. 

44 Visited during introduction, activity, practice, review and assessment. 
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46 Students were assigned to the library of computer lab. I tried grouping students 
according to Test Item Bank results. 

48 Try to get groups of kids on PLATO Elementary beginning and ending of class. 



ELEMENTARY SCHOOL #1 

50 I am just trying to get students to be able to log on and log off and follow the 
directions at this point. (No computer lab available.) 

51 The children rotate once a week to work on the computer for 30 min. (Sometimes 
they occasionally work on it in the afternoon.) 

52 My students use PLATO Elementary during the reading block for one of their centers. 
It is used for practice and reinforcement of skills. 

54 Mostly during the practice, review, or transfer (application) phase? What I’m seeing 
is that it is used as a “center” without prior explanation or connection. 

55 Practice, review, and application phase. 

56 Students regularly use during reading and math. 

57 PLATO Elementary is used as reinforcement. 

58 DVES [*] — Computer lab on a weekly basis. 

60 What lab. 

63 After regular lessons are done. 

64 It varies. 

70 No. 



4. Describe any strategies you employed to determine whether or not the PLATO 
Elementary activities assigned to each student were the most appropriate for ensuring 
their success in your class. 

MIDDLE SCHOOL #1 

01 I tried doing some of the activities myself. I would sit by students and help if needed. 

05 I weave lessons from text and PLATO Elementary together and evaluate the mix. 

06 I monitored and discussed with them what they liked or disliked. 

08 I assigned them to myself and checked it out. 

11 I use PLATO. 

16 I worked the lesson as a student. 

17 Still working on that. 

MIDDLE SCHOOL #2 

Veteran Teachers 

20 I previewed the assignments. 

23 Checking and aligning codes. 

24 Match the proficiency codes. 

25 Relate PLATO Elementary to mastery or not from test item bank. 
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26 I viewed them myself. 

31 Viewing them. 

33 We talked about how we applied the information in class, to the assignments. PLATO 
Elementary constantly froze up, made them repeat assignments they had completed, etc. 

34 Preview them myself. 

35 Instruction — follow in class. 

37 Went through them myself. 

39 Pulling assessments — retesting after completion of PLATO Elementary. 

New Teachers 

40 Looked at them first. 

42 I preview each assignment. 

43 Preview. 

44 I review every assignment before they are required to start or complete it. 

46 None — I’m still confused. 

48 Tried to preview before assigning but difficult to manage time. 

ELEMENTARY SCHOOL #1 

5 1 I used the PLATO Elementary programs that were recommended at the inservices. 

52 I’m confused about the success of my students on PLATO Elementary because the 
report indicated that one student was on PLATO Elementary for 20 seconds and his score 
was 1 00%. How can that be? 

54 I see a lot of students who are not placed at their instructional levels or application 
levels. 

55 Monitored if extra practice improved ability on a particular skill. 

57 Use the standards. 

60 Use of peers. 

62 I matched them to the proficiencies and to their own level. 

63 Whether activities matched the objectives on their IEPs [Individual education plans]. 

64 Pre-testing and scoring reports. 

70 Peer tutoring. 



5. How would you change the PLATO Elementary lessons? 

MIDDLE SCHOOL #1 

01 Maybe make the PLATO Elementary Units apply to a whole curriculum unit. Now 
you have to click on parts of a PLATO Elementary unit for the appropriate codes. 

02 Make them less elementary. 

03 I wouldn’t, they are great. 

08 I’m not sure if you could create a computer based lesson that would do what I need it 
to do in Language Arts. 

09 I wouldn’t. It works for most students. 

1 1 Be more direct. 
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16 I wouldn’t. 

1 7 More U.S. history software. 

1 8 Update them. 

MIDDLE SCHOOL #2 
Veteran Teachers 

19 Add real and relevant social studies lessons tied to state standards 

20 They need to be more challenging. 

25 Not on Internet. 

26 I wouldn’t, but I would be more lenient about using PLATO as a substitute until the 
concepts are learned. 

27 Add some music specific — maybe theory, history, ear training — topics. 

28 I need more computer time for my classes. 

30 Design a PLATO Elementary “class” for new students and sixth graders and low kids. 

33 Kids are confused by the phone. They’re unclear on how to use it. I think they don’t 
value it or its use. 

34 Lessons are OK. 

36 Nothing. 

37 Increase physical science concepts — chemistry and physics. 

New Teachers 

40 More objective-specific. 

41 More lessons on the objectives. 

43 Make them a bit longer (each lesson only lasts about 1 5 minutes). 

44 I would not make any changes at this time. 

48 Make them easier to work with and understand. Easier to go through program. 
ELEMENTARY SCHOOL 

50 I would have the (show-next) icons blink when ready. Kids who can’t read are 
expected to read those buttons, too. 

54 If the lessons were designed with the sub-skills broken out before so students could 
review. (For example, in the “Yellowstone connections” project, kids need to know bar 
graphics and how to multiply, e.g., numbers with zeros.) 

57 Not enough objectives . Need teacher time. 

63 More pre/post tests to see whether student needs or has skills in the lessons. 

64 1 would make sure all of the codes match and all district codes were offered. [From 
comments written on Part 1 question #4: Content seemed generally free of errors and 
inaccuracies.] Codes are sometimes mismatched and program is slow. 

70 More time blocks. 



6. What suggestions do you have to improve the way you use the PLATO Elementary 
system? 



Apache Junction, AZ 



Page 43 



MIDDLE SCHOOL #1 



01 Have more computers available that work. 

03 Separate it from Internet Explorer. Make it a stand alone system. 

06 When every computer WORKS (which is not very often), all is well! 

08 I don’t think I need to improve on the way I use PLATO Elementary. I feel I am the 
teacher, and this should just be used as a reinforcement tool, not a way of teaching. 

09 I could learn to use it more in reading and language. 

1 1 I suggest we do not use a system that is “cold” and complicated and requires 
computers we do not have! 

MIDDLE SCHOOL #2 
Veteran Teachers 

20 Gain more knowledge of the system. 

25 Delete old assignments. 

29 Allow more time for the students. 

31 More whole class use — labs. 

33 I would rather not use the system for Language Arts. Perhaps social studies will be 
easier for kids to understand and use. 

34 Less icons — a tutorial lesson on just how students use the system — they go in without 
knowing how to navigate. 

39 Need more computers — tough to get all 35 on throughout week. 

New Teachers 

43 Reduce technical problems. 

44 I would like to have computers in my room so I can apply the PLATO Elementary 
system more effectively on a more consistent basics. 

48 None — first year — just trying to survive. 

ELEMENTARY SCHOOL 
50 Improve the computers themselves. 

54 Tying to lessons they’re doing in regular class. More discussion beforehand. Higher 
order thinking skills for the kids (train them). 

58 Teachers need more time to assign lessons, assess reports, and utilize the program to 
the best of our advantage. 

60 Not enough time in the day to use. 

64 I would make sure all of the codes match and all district codes were offered. 



7. What other comments or suggestions do you have on the PLATO Elementary system 
or this course? 

MIDDLE SCHOOL#! 
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01 lam happy that PLATO Elementary has a new server. I am ready to try it again this 
year with my students. [On front page]: When assigning lessons by the curriculum codes, 
sometimes the PLATO Elementary activities did not match up complete with the 
objective. For example, the “setting” of a story has only one question on the setting, but a 
whole activity on writing a script for TV 

06 It just seems that something could be done to increase the reliability factor — 
program=accessible; computers=operable 

08 I feel the students should have a voice when this is concerned. Ask them how much 
they actually understand or learn from PLATO Elementary. 

10 Have more subjects! Even items for ESL. 

1 1 We need to consider the topic of access — I have classes of 33-36 and it is hard to 
schedule into a lab that has enough computers working. 

14 Don’t like to use PLATO Elementary — prefer PLATO. 

MIDDLE SCHOOL #2 

Veteran Teachers 

1 9 Could be more user friendly. 

20 There needs to be more of a connection between student interests and the 
assignments. 

22 Better training. 

24 It’s OK. 

31 Have the kids go through the training. 

32 Many students had difficulties determining what was being asked by the question, 
which frustrated them. 

33 [Trainer] needs to be a little more patient in his presentations. He is very short and 
rude. We’re adults. Treat.us as such. [From comments written on Part I question #6 — 
Quality and style of instruction was consistent throughout the curriculum.]: Quality — No 
good in language arts; consistent — yes, but not good. 

34 Technology questions. 

35 [From comments written on Part 1 question #17 — I was able to spend time in one-on- 
one tutoring...] Too much time trouble-shooting software. 

[From comments written on Part I question #24 — I find working with the computer is 
generally a productive, rather than frustrating, experience.]: Once bugs are ironed out on 
PLATO Elementary, this may be the case. 

37 Increase physical science concepts. 

New Teachers 

40 Easier. Less time-consuming. Computer does work for you. 

44 I think it is a great tool that enhances the teaching/leaming experience. 

46 Give me more training — I need help in lessons, reports, and time management. 

48 Get bugs out so program runs smoothly consistently. Have all plug-ins loaded and 
ready to go. 

49 This class was very informative. Thanks for your help! Item 23: [My students] voice 
concern of it being childish. 
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50 We all just need more practice! ! 

52 [Added on other items]: Students cannot understand log out directions. 

54 Kids I see are generally guessing on answers — not thinking about them. It becomes a 
“guessing game” and I’m not sure how well they’re using their time. I have more one-on- 
one time with them because I work with just PLATO Elementary kids. In order for the 
projects to really be effective I stay right with them. [She works in the library.] 

56 Are there lessons for intermediate kids that are at a lower reading level? Most 
intermediate objectives have grade-level vocabulary which my lower kids can’t read 
independently. 

57 Make logins easier for primary. 

58 [From comments written on Part I question #23: [Students] were bored by Christmas. 
61 [From comments written on Part II question #33: Clearly identified to students 
rewards and incentives...] All they got was a point! 

64 We need to have more time to have students on; lessons take awhile. 

70 Less trouble-shooting. 
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